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SUBSTITUTE SPECIFICATION 

TITLE OF THE INVENTION 

CODING AND DECODING APPARATUS USING TOOL INFORMATION FOR 
CONSTRUCTING A DECODING ALGORITHM 

This application is a Divisional of co-pending Application 
No. 09/407,880, filed on September 29, 1999, which is a 
Divisional of prior Application No, 08/727,787, filed on 
October 8, 1996 and issuing as U.S. Patent No. 5,987,181 on 
5 November 16, 1999, the entire contents of which are hereby 
incorporated by reference and for which priority is claimed 
under 35 U.S.C. § 120; and this application claims priority of 
Application Nos . 7-264127 filed in Japan on October 12, 1995; 
7-277463 filed in Japan on October 25, 1995; 7-308915 filed in 
10 Japan on November 28, 1995; and 7-310667 filed in Japan on 
November 29, 1995 under 35 U.S.C. § 119. 

BACKGROUND OF THE INVENTION 

(1) Field of the Invention 

The present invention relates to a coding and decoding 

15 apparatus for coding information such as an image signal etc. 
to transmit it and decoding the coded data. More detailedly, 
the present invention relates to a coding and decoding 
apparatus which enables the communication between the coding 
and decoding tools having different processing capacities and 

2 0 in which the coding apparatus transmits not only the coded data 
but also coding information for the construction of a decoding 
scheme as the means of decoding the coded data and the decoding 
apparatus receives the coding information together with coded 



- 2 - 



data and reconstructs the decoding scheme based on the coding 
information so as to decode the received coded data. Further, 
the present invention is directed to a coding and decoding 
technology for performing the communication in a coding and 
5 decoding apparatus between the transmitting and receiving 
devices having different capacities in the case where the an 
algorithm includes various coding and decoding tools such as 
near- future image coding schemes represented by the MPEG4 etc., 
and more particularly relates to a coding and decoding 
10 apparatus which enables simultaneous transmission of coded data 
and tool information for constructing the algorithm for 
decoding the coded data in order to realize a hierarchical 
coding and decoding operation. 

(2) Description of the ftejreaeBackground Art 
15 In recent years, a wide spread of ISDN (Integrated 

Services Digital Network) has realized image communication 
services as a new communication service. Examples of the 
services include the video phone and video conference system, 
etc. On the other hand, the development of the mobile 

2 0 communication networks represented by the PHS and the FPLMTS, 

accelerates demands for further betterment improvement and 
variations of the services and portability of the devices. 

In general, in the case where image information as in the 
video phone or video conference system is transmitted, the 
25 amount of image information is very large. However, due to the 
line speed used for the transmission and the cost problem, the 
image information to be transmitted needs to be compressed and 
coded so that the amount of information can be reduced. 

As to the coding schemes for compressing image 

3 0 information, JPEG (Joint Photographic coding Experts Group) has 

already been standardized internationally for a still image 
coding system, H.261 for a motion picture coding scheme, and 
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MPEG1 (Moving Picture Coding Experts Group) and MPEG2 for motion 
picture coding schemes. Further, MPEG4 is now being 
standardized as a coding scheme of very low-bit rate of 64 kbps 
or below. 

5 In the current coding schemes such as JPEG, H.2 61, MPEG1, 

MPEG2, coding is performed following the specified algorithm. 
However, the MPEG4 is planned to flexibly deal with various 
applications and encode each of the applications in its optimal 
scheme. For this purpose the MPEG4 needs to have many tools 

10 (such as transformer, quantizer, inverse transformer, inverse 
quantizer, etc.) for its coder so that a suitable combination 
of them will be selected to perform coding. 

Fig.lA is a conceptual view showing the structure of a 
coding data stream which is formed by coding (compressing) 

15 image data based on the H.261 scheme. Each piece of the coded 
data such as motion vector information, DCT-coef f icient , 
quantization step, etc., shown in Fig.lA is image data which 
has been coded (compressed) based on a fixed coding algorithm 
in the coder, while the decoder has a decoding algorithm fixed 

2 0 corresponding to the coding algorithm so that the received 

pieces of the coded data will be decoded. 

Fig. IB is a conceptual view showing the structure of a 
coding data stream which is formed by coding (compressing) 
image data based on a coding scheme such as MPEG4 etc . whose 
25 algorithm is flexible. The coding data stream as shown in 
Fig. IB is composed of coded (compressed) image data such as 
motion vector information 2, transform coefficient 4, motion 
vector information 6, transform coefficient 8 and quantization 
step 10 etc., and tool information such as motion compensation 

3 0 tool 1, inverse transform tool 3, motion compensation tool 5, 

inverse transform tool 7 and quantizing tool 9, etc., for 
decoding respective image data. Fig. IB illustrates the details 
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of the motion vector information, DCT-coef f icient and 
quantization step at the leading end of the coding data stream 
of Fig. 1A. In this case, each piece of the tool information 
such as motion compensation 1 etc., is allowed to be selected 
5 from a number of types of the tool information so that it is 
possible to freely select a desired combination of the tool 
information. Accordingly, the coder transmits the tool 
information which has been used for coding as well as the image 
data to the decoder. The decoder, upon the decoding of the 

10 image data received, will decode the coded image data using the 
tool information transmitted from the coder. 

Fig.lC is a block diagram showing an example of a 
conventional coding and decoding apparatus based on H.261. This 
coding and decoding apparatus is composed of a controller 6a 

15 for controlling the entire apparatus, a coder 7a for coding 
based on H.2 61, and a decoder 8a for decoding the information 
which has been coded based on H.261, and a tool storage section 
9a consisting of memories for storing tool information. 

These coding and decoding processes can be realized by a 

2 0 dedicated hardware device with software installed therein or by 
an appropriate program executed in a general -purpose processor 
with a compiler. 

First, description will be made of a method using a 
dedicated hardware device with software installed therein. 

2 5 Fig. 2 is a block diagram showing the configuration of coder 7a 
of Fig.lC for yielding the coded data shown in Fig.lA, based on 
H.261. In Fig. 2, the coder is composed of: a coding controller 
11 for the control of coding; a transformer 12 for performing 
the DCT; a quantizer 13 for quantizing the coefficients 

30 transformed by transformer 12; an inverse quantizer 14 for 
performing inverse quantization of the coefficients quantized 
in quantizer 13; an inverse transformer 15 for performing the 
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inverse DCT; a memory 16; and a loop filter 17. Here, memory 16 
ia the one which has the function of causing a variable delay 
for motion compensation, used when the inter- frame prediction 
for motion compensation is performed. Filter 17 is the loop 
5 filter capable of performing the on/off operation for each of 
macro blocks . 

When the coding algorithm for generating the coding data 
stream shown in Fig.lA is executed by the dedicated hardware 
device with software, the tool functions constituting the 

10 algorithm are carried out by software and the dedicated 
hardware components as shown in Fig . 2 , namely, coding 
controller 11, transformer 12, quantizer 13, inverse quantizer 
14, inverse transformer 15, memory 16 having the function of 
causing a variable delay for motion compensation, and loop 

15 filter 17. Fig. 3 is a block diagram showing the configuration 
of decoder 8a shown in Fig.lC for decoding the coded data based 
on H.261. This decoder commonly has the constituents of the 
coder shown in Fig. 2, and the same components as those in the 
coder of Fig. 2 are designated at the same reference numerals. 

20 Specifically, in Fig. 3, a reference numerals 14 designates an 
inverse quantizer, 15 an inverse transformer, 16 a memory 
having the function of causing a variable delay for motion 
compensation, and 17 a loop filter. 

The coded data by the coder shown in Fig. 2 is inverse 

25 quantized by inverse quantizer 14, and the signal is then made 
to undergo the inverse DCT in inverse transformer 15. Here, 
memory 16 and loop filter 17 are used when the motion 
compensated prediction coding data is decoded. 

When several kinds of algorithms need to be processed 

3 0 using the scheme which performs the coding operation based on a 
fixed algorithm such as H.2 61 etc. as stated above, an 
individual hardware device with software is needed to execute 
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each of the algorithms. Fig . 4 is block diagram showing the 
structure of a coder which codes the signal of a motion picture 
based on H.2 61 and the signal of a still image based on JPEG. 
For example, when a motion picture is coded based on H.261 and 
5 a still image is coded based on JPEG, the coder should have the 
configuration as shown in Fig. 4, which includes two individual 
coders, namely a H.2 61 coder 2 0 and a JPEG coder 21. In Fig. 4, 
H.261 coder 20 and JPEG coder 21 receive the motion picture 
data and the still image data respectively to output coded data 

10 of compressed data. 

When the algorithm for generating the coded data shown in 
Fig. IB is executed by a dedicated hardware device with 
software, a coder for executing this algorithm is realized by 
the one shown in Fig. 2 in which the circuit block designated at 

15 18 is configured by the configuration shown in Fig. 5. In this 
case, the coder has plural types for each of the tools, or, 
transformer 12, quantizer 13, inverse quantizer 14, and inverse 
transformer 15. In this configuration, one desired type is 
selected for each of the tools (one type from transformer tools 

2 0 A to X, one type from quantizer tools A to X, one type from 

inverse quantizer tools A to X and one type from inverse 
transformer tools A to X) to perform a coding process. 

The decoder for decoding the coding data stream shown in 
Fig. IB is realized in a decoder shown in Fig. 3 in which the 
25 circuit block designated at 19 is replaced by a circuit block 
22 in Fig. 5. In this case, the decoder has plural types for 
each of the tools, or, inverse quantizer 14, and inverse 
transformer 15. In this configuration, one desired type is 
selected fore ach of the tools (one type from inverse quantizer 

3 0 tools A to X and one type from inverse transformer tools A to 

X) to perform a decoding process. 



- 7 - 



In this decoding process, each piece of the tool 
information shown in Fig. IB, for motion compensation tool 1, 
inverse transforming tool 3, motion compensation tool 5, 
inverse transforming tool 7 and quantizing tool 9 is sent to a 
5 controller 23, and each piece of the image data, which follows 
the corresponding tool information, specifically, of motion 
vector information 2, transform coefficient 4, motion vector 
information 6 and transform coefficient 8, is sent to the 
corresponding tools where each image data is processed- At the 

10 time, controller 23 selects one of the tools (one from inverse 
quantizing tools A to X and one from inverse transforming tools 
A to X shown in Fig. 5) based on the corresponding tool 
information, in this way, each piece of the image data is 
processed through the tool selected by controller 23 and is 

15 decoded. 

However, this method needs a dedicated device with 
software for each of the tools, thus the scale of the decoder 
tends to become large. To make matters worse, if the decoder 
receives the data which has been processed by a tool that is 

2 0 not provided for the decoder, it is impossible to decode the 

data itself. To solve this problem, a way that can be 
considered is one in which parts received should be compiled to 
prepare a processing program and the data should be decoded by 
a general -purpose processor. 
25 Next, description will be made of a method of achieving 

the decoding process by executing a suitable program using a 
general -purpose processor with a compiler. Now, referring to 
Fig. 6, description will be made of a case where the coding data 
stream having the structure shown in Fig. IB is decoded. Fig. 6 

3 0 is a block diagram showing the structure of a decoder composed 

of a general -purpose processor 24 and a compiler 25. When all 
the tool information as shown in Fig. IB, which includes a 
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motion compensation tool 1, inverse transforming tool 3, motion 
compensation tool 5, inverse transforming tool 7 and quantizing 
tool 9, etc., is given to compiler 25, the compiler will 
prepare a processing program for controlling the operation of 
5 general -purpose processor 24. Each piece of the image data, 
which follows the corresponding tool information, specifically, 
motion vector information 2, transform coefficient 4, motion 
vector information 6, transform coefficient 8, quantization 
step 10, is given to general -purpose processor 24. Then, 

10 general -purpose processor 24 processes, with the processing 
program prepared by the compiler 25, the coded image data 
following the tool information so as to decode it for producing 
its decoded data. 

In the case where the capacity of the decoding apparatus 

15 for processing a certain algorithm is lower than the total 
processing capacity for all the tools constituting the 
algorithm requested by the coder side, even if the tools 
transmitted from the coder is stored at the decoder side, the 
received data cannot be decoded exactly due to the inferior 

20 processing capacity of the decoder side. Thus the memory in the 
tool storage is also used just in vain. 

Also in the conventional coding and decoding apparatus, 
when the tools which were used in the coder side are compared 
to the tools which are stored at the decoder side, the tools 

25 themselves should be compared to each other; this process 
required a very long period of time. 

In the case where a new algorithm is used to decode the 
coded information, even if the tools for the algorithm are 
equivalent to those which have been previously stored, the 

3 0 decoder should receive the tools once again; this process also 
considerably lengthened the transmission/recept ion time. 



- 9 - 



In this way, when a video signal etc. is coded, the coding 
tools having suitable coding capacities to the quality of the 
reproduction image required by the decoder side, are selected 
to perform the coding operation. When the thus obtained coded 
5 data is decoded, it is necessary that the decoder should use 
decoding tools having decoding capacities (i.e. processing 
capacities) which correspond to the coding capacities (i.e. 
processing capacities) for the coding tools which were used for 
the coding operation. Processing capacity indicates a resource 

10 that is necessary for coding, decoding or both. For example, 
coding capacity may be expressed as processing capacity for 
coding. If these tools on the decoder side do not have the 
processing capacities for the tools on the coder side, the 
coded data cannot be decoded, thus making it impossible to 

15 establish the communication. 

An example of the algorithm for the inter frame predictive 
coding will hereinbelow be described. Frame predictive coding 
shall mean inter- frame predictive coding, intra-frame 
predictive coding, or both as the context requires. Inter-frame 

2 0 predictive coding refers to any technique for data compression 
in which a subsequent frame, or a portion thereof, is encoded 
as differential data with respect to an earlier reference 
frame. Pixel data may be expressed as such differential data if 
inter-frame predictive coding is used. Illustratively, 

2 5 description will be made of the influence on the communication 
when the processing capacities for the inter frame predictive 
coding tools are not in agreement with those for the inter 
frame predictive decoding tools. The inter frame predictive 
coding aims the improvement of may be considered as improving 

30 the quality of a display image on the decoder side and since 
inter- frame predictive coding is an image data processing 
technology by which, — based . Based on the data of the pixels 



- 10 - 



directly obtained by sampling the video signal on the coder 
side, the pixel data for the display pixels on the decoder side 
which are defined more minutely than the sampling pixels of the 
dccodcr coder side-? — will be since the pixels on the decoder side 
5 are predictively interpolated. 

Figs . 7A through 7C are conceptual diagrams for 

illustrating feke principle fehe inter pixel data 

arrangements resulting from frame predictive coding. Figs.7A, 
7B and 7C show the arrangements of pixel data (a x to di, a 2 to 

10 ± 2 , a 4 to Y 4 ) produced based on the image data A to D which is 
directly obtained by coding the video signal inputted from a 
visual sensor such as a camera etc., by means of the inter 
frame predictive coding tools of sampling the data per single 
pixel, per M pixel and per M pixel, respectively. This pixel 

15 data is transmitted as the coded data obtained by the inter 
frame predictive coding, from the coder side to the decoder 
side. Here, in each of the inter frame predictive coding tools, 
the arithmetic operation for each piece of the pixel data is 
made based on the calculating formulae shown in Table 1. 



20 
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In Fig7A, pixel data ai to di for the pixels indicated by 
' + ' is the pixel data (corresponding to the (n+1) -ranked coded 
data when the pixel data obtained by the aftermentioned inter 
5 frame predictive coding tool of sampling per M pixel is assumed 
as the n-ranked coded data) produced by the inter frame 
predictive coding tool (corresponding to the (n+1) -ranked 
coding tool when the aftermentioned inter frame predictive 
coding tool of sampling per M pixel is assumed as the n-ranked 
10 coding tool) of sampling per single pixel. In this case, pixel 
data ai to di obtained by the inter frame predictive coding is 
equivalent to image data A to D which is directly obtained by 
coding the video signal . 
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In Fig.7B, pixel data a 2 to i 2 for the pixels indicated by 
' + ' and 'o' is the pixel data (corresponding to the (n+1) - 
ranked coded data when the pixel data obtained by the 
af termentioned inter frame predictive coding tool of sampling 
5 per y A pixel is assumed as the n-ranked coded data) produced by 
the inter frame predictive coding tool (corresponding to the 
(n+1) -ranked coding tool when the af termentioned inter frame 
predictive coding tool of sampling per V* pixel is assumed as 
the n-ranked coding tool) of sampling per 3^ pixel. Of these, 

10 the pixel data for the pixels indicated by ! + ' is equivalent to 
the pixel data obtained by the inter frame predictive coding 
tool of sampling per single pixel, while the pixel data for the 
pixels indicated by 'o 1 is the interpolated pixel data which 
has been predicated based on image data A to D. 

15 In Fig.7C, pixel data a 4 to y 4 for the pixels indicated by 

' + ' 'o', 'A 1 is the pixel data produced by the inter frame 
predictive coding tool of sampling per X A pixel. Of these, the 
pixel data for the pixels indicated by 1 + ' and 'o' is 
equivalent to the pixel data obtained by the inter frame 

20 predictive coding tool of sampling per % pixel. The pixel data 
for the pixels indicated by ■ + ■ is equivalent to the pixel data 
obtained by the inter frame predictive coding tool of sampling 
per single pixel, while the pixel data for the pixels indicated 
by ! o f and 'A 1 is the interpolated pixel data which has been 

25 predicated based on image data A to D. 

As understood from Figs . 7A to 7C, pixel data a 2 to i 2 
obtained by the inter frame predictive coding per M pixel 
sampling, includes pixel data ai to di (image data A to D) 
obtained by the inter frame predictive coding per single pixel 

30 sampling, therefore the pixel data is distributed four times as 
dense as that of sampling pixels of the video signal. Pixel 
data a 4 to y 4 obtained by the inter frame predictive coding per 
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V A pixel sampling, includes the pixel data obtained by the 
inter frame predictive coding per single pixel sampling and per 
% pixel sampling, therefore the pixel data is distributed 
sixteen times as dense as that of the sampling pixels of the 
5 video signal. 

In this way, the pixel data obtained by the inter- frame 
predictive coding tools for producing the pixel data of high- 
density display pixels, hierarchically includes the pixel data 
obtained by the inter frame predictive coding tools for 

10 producing the pixel data of the lower density display pixels. 
For instance, pixel data a 4 to y 4 obtained by the inter frame 
predictive coding tool of sampling per y A pixels hierarchically 
includes pixel data a 2 to i 2 as well as pixel data a x to d x . 

Figs.8A to 8C are illustrations explaining the effects on 

15 the image display by the inter- frame predictive coding and 
showing the display images of a pattern TA obtained by decoding 
the coded data based on the inter- frame predictive coding per 
single pixel sampling, per ^ pixel sampling and per M pixel 
sampling, respectively. In Figs. 8A to 8C, ' o' and '•' 

2 0 represent pixels of 'light' and 'dark' display states, 

respectively when the coded data obtained by subjecting the 
video signal of pattern TA as a subject to the inter- frame 
predictive coding is decoded. Here, in Figs.8A to 8C, pattern 
TA as a subject is assumed to move in the lower right 
25 direction, and for easy understanding of the positional 
relation between pattern TA and the pixels, display images are 
laid over the pixels, for reference. 

Referring to Figs . 8A through 8C, according to the inter 
frame predictive coding per single pixel, pattern TA is 

3 0 represented wit three pixels before movement and with one pixel 

after movement. On the other hand, according to the inter frame 
predictive coding per X pixel, pattern TA is represented with 
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six pixels before movement and with three pixels after 
movement. Further, according to the inter frame predictive 
coding per M pixel, pattern TA is represented with fifteen 
pixels before movement and with ten pixels after movement. In 
5 this way, as the dividing number of the pixels in the inter 
frame predictive coding is increased so that the density of the 
display pixels at the decoder side is increased, it becomes 
possible to reproduce an image of high quality with high 
precision . 

10 Next, description will be made of a means for practicing 

the decoding scheme by the combination of individual functional 
tools (functional modules) independent of one another in the 
coder described above. 

Fig. 9 shows an example of a coding data stream to be used 

15 when the coded data based on H.261 is transmitted to a device 
which does not have the decoding function based on H.261. As 
stated above, since it is assumed that the coding scheme is not 
invariant and the combination of the functional tools in the 
coder can be freely selected, it is necessary to transmit the 

2 0 information of the type of the coding scheme based on which the 

signal was coded and the types of the functional tools used in 
the coding process (this information will hereinbelow be 
referred to as coding information) , together with the coded 
data. In Fig. 9, the data stream includc includes : coding 
25 information composed of motion compensation tool 112a, inverse 
transforming tool 112b, quantizing tool 112c and decoding 
scheme constructing information 111; and coded data of motion 
vector information 113a, transform coefficient 113b and 
quantization step 113c, which follow the corresponding coding 

3 0 information. The aforementioned each of the functional tool 

112a to 112c designate the orders of decoding corresponding 
coded data 113a to 113c, and may contain operation 
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specifications in some cases, may just indicate the identifying 
numbers of the functional tools in the other cases. Decoding 
scheme constructing information 111 specifies the functional 
tools to be used and the methods of using the resultant outputs 
5 from the tools, and other information. In the case shown in 
Fig. 9, the result after the motion compensation is used to 
handle the data of a certain image block decoded right before, 
for instance. That is, this result indicates the information 
relating to the order of procedure of the coding scheme H.2 61 

10 in this case. The device on the decoding side, which has 
received the coding data stream shown in Fig. 9, is able to 
construct the decoding scheme by interpreting the decoding 
scheme constructing information, the motion compensation tool, 
the inverse transforming tool and the quantizing tool so that 

15 it can exactly decode the decoded data that follows. 

As stated above, the coding information may contain the 
processing order of the tools and how to use the result 
obtained from each tool etc., so that the decoder will be able 
to decode the received coded data even if the signal which 

2 0 requires tools or is based on a decoding scheme that is not 
provided on the decoder side, is received. In order to improve 
the efficiency in the use of the line, however, it is 
preferable to use a decoding scheme which is able to work with 
a less amount of data to be transmitted such as the 

2 5 specifications on the construction of the decoding scheme and 

the tool information. In practice, since the purpose of the 
usage and the required quality will be determined to a certain 
degree depending on the coding and decoding apparatus, it is 
realistic that each coding and decoding apparatus has a number 

3 0 of coding and decoding schemes, in advance, which are expected 

to be used more frequently. 
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Fig. 10 shows an example of the coding data stream which 
can be used for the communication between two devices both 
having some coding and decoding schemes which are expected to 
be used more frequently. For the coding information, the same 
5 decoding scheme incorporated in the decoder is called up by 
transmitting a predetermined identification code 121a so that 
the coded data received can be decoded. Comparing with the 
example of Fig. 9, since this method will not need the 
transmission of the information on functional tools and the 

10 decoding scheme constructing information, it is possible to 
drastically reduce the transmitted amount of data and therefore 
the improvement of the efficiency in the use of the 
communication line can be expected. 

However, if the divided number of pixels in the inter 

15 frame predictive coding is dissimilar (or the inter frame 
predictive coding tools are different) , the structure of the 
coded data becomes quite different and thus it bcGomo becomes 
impossible to interchange the coded data. For this reason, in 
accordance with the conventional coding and decoding system 

2 0 (method and devices) , the decoding side needs to perform its 
decoding operation using a decoding tool suitable to the 
structure of the coded data. That is, the decoding tool should 
have the decoding capacity in one-to-one correspondence to the 
coding capacity for the coding tool to perform the decoding 

2 5 operation. Therefore, when the processing capacity for the 
decoding tool is not in agreement with that for the coding 
tool, it is totally impossible to decode the coded data 
received. 

When the data which is coded using an algorithm provided 
30 with various tools (represented by MPEG4 , for example) is 
tricd at tempted to be decoded by the device just having a single 
algorithm such as MPEG1 , the decoding side needs additional 
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hardware and/or software for operating the algorithm (coding 
tools) used in the coding. Therefore, the device is increased 
in size and cost. 

As seen also in the H.261 coding scheme etc., the detailed 
5 specifications of the coding scheme is usually switched 
depending on which is made more account of more important , the 
efficiency of coding or the quality of image, or depending upon 
the nature etc., of the input image. Further, the usage will 
be limited if the system has only limited types of coding 

10 schemes previously equipped, as stated above. Therefore, it 
becomes necessary to change over the coding scheme in 
accordance with the usage. In this case, if the coder side 
tries to transmit data coded based on the scheme that is not 
equipped on the decoder side, the coding information should be 

15 simultaneously transmitted, as already mentioned above. At 
this moment, in accordance with the aforementioned method, all 
the coding information, as shown in Fig. 9, including the 
information of the functional tools used in the decoding scheme 
previously provided in the decoder side, needs to be 

2 0 transmitted regardless of whether the difference of the 
decoding scheme from the coding scheme is small or great. That 
is, even when the coding scheme is not much different from the 
decoding scheme that is previously provided, the communication 
may require a large transmission rate, thus possibly reducing 

25 the efficiency in the use of the line. In practice, however, 
since there are some functional tools which can be commonly 
used in with little dependence on the difference in coding 
schemes, such as the transform coding in the motion picture, 
etc., it is possible to develop different kinds of coding 

30 schemes by adding other functional tools to the basic 
functional tools as such. 
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Further, in recent years, it has become possible to 
download the tools for JPEG and MPEG1 stated above, on the 
personal computer communications network etc., and receive 
image signal and decode it based on the downloaded tools. 
5 Therefore, it can be guessed readily that in the near- future 
video communications, the communication will be able to be 
performed by downloading the tools for coding and decoding. 
However, in the aforementioned coding and decoding system of 
the conventional scheme, the communication can be performed 

10 based only on the limited kinds of coding and decoding 
algorithms. In the case of the next generation image coding 
scheme (such as MPEG4) which can flexibly deal with various 
applications and can code the signal in the most suitable 
manner to each of the applications, if several kinds of 

15 algorithms are tried to be processed by a scheme which performs 
coding with a fixed algorithm such as JPEG, H.261, MPEG1 , 
MPEG2 , etc., it becomes necessary to provide hardware and/or 
software for executing each algorithm. In this way, it is 
preferable that all the various kinds of algorithms are 

2 0 provided for both the transmitting and receiving sides. 
However, if all the tools are provided to deal with all the 
algorithms, the hardware and software becomes bulky, and the 
apparatus will increase in cost and inevitably becomes large. 
On the other hand, if the apparatus is reduced in cost and size 

2 5 and therefore the apparatus does not have adequate capacities, 
the risk of the failure to perform communications becomes high. 

In the coding and decoding apparatus which dooc does not 
have the above capacity, the decoder will download the tools 
for the required algorithm so as to be able to flexibly deal 

30 with the various kinds of applications and decode the signal. 
In such a coding and decoding apparatus which downloads the 
tools for the algorithm and is able to store the tools 
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previously used, if the tools stored are not the ones which are 
required for the next communication, the required tools must be 
downloaded again before the transmission of the coded data. 
Therefore, the delay before the start of transmission to the 
5 decoding of the coded data becomes long. 

In the above coding and decoding apparatus which is able 
to store the tools previously used, if the coding and decoding 
tools are provided in such a hierarchical manner that the tools 
for high quality is provided at the lower rank and the tools 

10 for assuring minimum quality which are not replaceable with 
other tools are provided at the higher rank, it becomes 
possible to decode the signal using those tools for minimum 
quality even if the capacity of the decoding apparatus is 
different from that of the coding apparatus. In this case, the 

15 delay before the start of transmission due to the downloading 
of the tool information can be eliminated, however it is 
impossible to decode the signal with the anticipated quality. 
In this case, when the signal is decoded with the anticipated 
quality, it is necessary to previously download the tools for 

20 the anticipated quality. Therefore, the situation is quite 
similar to the case where the tools are not provided in the 
hierarchical manner. That is, the delay before the start of 
transmission to the decoding of the coded data becomes long and 
therefore it is impossible to make use of the merit from the 

25 hierarchical structure of the tools. 

SUMMARY OF THE INVENTION 
The present invention has been achieved in view of the 
above problems. It is therefore an object of the invention to 
provide a coding and decoding apparatus which can select the 
30 most suitable tools based on the comparison with the tool- 
correspondent information and can perform the decoding 
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operation efficiently and quickly when the received coded data 
is decoded using the tools simultaneously received. 

It is another object of the invention to provide a coding 
and decoding apparatus which is able to decode the coded data 
5 even if the coding capacity of the coding tools on the coding 
side is not in agreement with the decoding capacity of the 
decoding tools on the decoding side so that the apparatus on 
the decoding side can be reduced in size and price. 

It is a further object of the invention to provide a 

10 coding and decoding apparatus in which, by transmitting only 
the differential information as to the functional tool and the 
decoding scheme which are already prepared in the decoding 
apparatus when the decoding scheme is transmitted, it is 
possible to designate and identify the necessary decoding 

15 scheme with fewer steps, so that the efficiency in the use of 
the communication line will not be deteriorated critically. 

It is still another object of the invention to provide a 
coding and decoding apparatus of the next -generation image 
coding scheme, in which the coding apparatus on the 

20 transmitting side simultaneously transmits the decoding tool 
information and the coded data when the decoding apparatus on 
the receiving side has no decoding tools which are requested by 
the transmitting side, thus making it possible to save the time 
which would be required for downloading only the tool 

25 information and thereby shorten the time before the start of 
the transmission of the coded data, compared with the 
configuration which initially transmits tool information only, 
and then downloads this information and thereafter starts 
decoding by using the thus downloaded decoding tools. 

3 0 In order to achieve the above object of the invention, the 

gist of the invention can be represented as follows:- 
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In a coding and decoding apparatus of tbea first aspect of 
the invention where the coder transmits coded data together 
with identifying information for identifying the means of 
decoding the coded data, and the decoder is capable of storing 
5 a plurality of decoding schemes so as to perform decoding based 
on one of the previously stored schemes, in order that the 
coded data and the information of the tools which constitute 
the algorithm as the means of decoding the coded data can 
simultaneously be transmitted, the received tools can be 

10 reconstructed into the algorithm and the received coded data 
can be decoded based on the algorithm-? — fch e. The coding and 
decoding apparatus comprises: a tool storage mcano for storing 
tools; a tool -correspondent information otoring mcano storage 
for storing the information corresponding to the tools; a 

15 comparing device mcano for comparing the tool -correspondent 
information received with the information stored in the tool- 
correspondent information storage otoring mcano ; and a selection 
controlling mcano controller for selecting the optimal tool from 
the tool storage mcano based on the result from the comparing 

20 device mcano to perform the processing with the selected one— 
and the . The apparatus is constructed such that the coded data, 
tool information [ [ , ] ] and tool -correspondent information are 
all transmitted simultaneously, and the coded data is decoded 
using the tool selected based on the received tool- 

25 correspondent information. 

5%eA second aspect of the invention resides in a coding 
and decoding apparatus having the first feature and is 
characterized in that the tool -correspondent information 
comprises the processing capacity of each tool, and the 

3 0 processing capacity of the received tool is compared to a 
decoding capacity stored in the tool -correspondent information 
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otoring mcana storage so that the tools whose capacities fall 
within the range of the decoding capacity are selected. 

3feeA third aspect of the invention resides in a coding and 
decoding apparatus having the second feature and is 
5 characterized in that the processing capacity of the tool is 
numerically represented and transmitted. 

TheA fourth aspect of the invention resides in a coding 
and decoding apparatus having the second feature and is 
characterized in that the tool -correspondent information 

10 otoring — mcana storage includes a decoding capacity storage 
section for setting up a decoding capacity of the decoding 
apparatus and storing it and a coding capacity storage section 
for storing each of coding capacities of the tools transmitted 
from the coding apparatus, and the comparing mcano comparator 

15 comprises a capacity comparator which compares the coding 
capacity with the decoding capacity so as to judge whether the 
transmitted tool is processible. 

5%*eA fifth aspect of the invention resides in a coding and 
decoding apparatus having the first feature and is 

2 0 characterized in that the tool -correspondent information 
comprises keys unique to different tools, and received keys are 
compared to the keys stored in the tool -correspondent otoring 
moano storage so as to select the corresponding tools and 
operate the selected tools. 

2 5 TheA sixth aspect of the invention resides in a coding and 

decoding apparatus having the first feature and further 
comprises a response controlling mcano cont roller for requesting 
the coding apparatus on the opposite side to transmit the tool 
information only when tool information is required. 

3 0 Next, in a coding and decoding apparatus of tbea seventh 

aspect of the invention where the coder transits coded data 
together with identifying information for identifying the means 
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of decoding the coded data, and the decoder is capable of 
storing a plurality of decoding schemes so as to perform 
decoding based on one of the previously stored schemes, the 
apparatus is characterized in that n-ranked (n: a positive 
5 integer) coded data which is produced using an n-ranked coding 
tool and decoded using an n-ranked decoding tool has a 
hierarchical structure which includes (n+1) -ranked coded data 
which is produced using an (n+1) -ranked coding tool and decoded 
using a (n+1) -ranked decoding tool, the coder having an n- 

10 ranked coding tool is composed of: a coding mcano coder which 
produces the n-ranked coded data using the n-ranked coding 
tool; and an identifier adding — mcano adder which attaches N- 
ranked identifiers (N: a positive integer satisfying N > n) to 
N-ranked coded data which is included in the n-ranked coded 

15 data but is other than (N+1) -ranked coded data included in the 
N-ranked coded data, and the decoder having an m- ranked (m is a 
positive integer satisfying m > n) decoding tool is composed 
of: a data rccono true ting mcano reconstructer which extracts 
the N-ranked coded data which is attached with the N-ranked 

2 0 identifiers where N > m, from the n-ranked coded data; and a 
decoding mcano decoder which decodes the m- ranked decoded data 
using the m-th decoding tool . 

ThcA n eighth aspect of the invention resides in a coding 
and decoding apparatus having the feature of the seventh aspect 

2 5 and is characterized in that the coding tool is an inter frame 

predictive coding tool and the decoding tool is an inter frame 
predictive decoding tool . 

Further, in a coding and decoding apparatus of thea ninth 
aspect of the invention where the coding side transmits coded 

3 0 data together with identifying information for identifying the 

means of decoding the coded data, and the decoding side is 
capable of storing a plurality of decoding schemes so as to 
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perform decoding based on one of the previously stored schemes, 
the coding and decoding apparatus is characterized in that when 
the coded data and the coding information which includes a 
decoding scheme as the means of decoding the coded data and 
5 functional tools constituting the decoding scheme are 
simultaneously transmitted, the decoding side receives the 
coding information and reconstructs the decoding scheme based 
on the coding information received, and the received coded data 
is decoded based on the reconstructed decoding scheme, an 

10 identification code of a previously defined basic decoding 
scheme and the differential information from the basic decoding 
scheme are transmitted as the coding information from the 
coding side so that the decoding side will recognize the 
decoding scheme required therefor. 

15 TheA tenth aspect of the invention resides in a coding and 

decoding apparatus having the ninth feature and is 
characterized in that the coding apparatus comprises: a 
database of coding schemes for storing plural kinds of coding 
schemes and functional tools which constitute the coding 

2 0 schemes; a coding scheme selector for selecting the coding 
scheme based on input data; a coding section for performing a 
coding process of the input data in conformity with the 
determined coding scheme; and a coding controller for 
controlling each section. 

2 5 TfeeAn eleventh aspect of the invention resides in a coding 

and decoding apparatus having the ninth feature and is 
characterized in that the decoding apparatus comprises: a 
database of decoding schemes for storing plural kinds of 
decoding schemes and functional tools which constitute the 

3 0 decoding schemes; a decoding scheme constructing section for 

reconstructing the decoding scheme in accordance with the 
received coding information; a decoding section for performing 
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a decoding process of the received data in conformity with the 
reconstructed decoding scheme; and a decoding controller for 
controlling each section. 

3 1 heA twelfth aspect of the invention resides in a coding 
5 and decoding apparatus having any one of the ninth through 
eleventh features and is characterized in that the 
identification code of a basic decoding scheme and the 
information that one or some kinds of functional tools will be 
added to the basic decoding scheme, are transmitted as the 

10 coding information, so that the decoding scheme incorporated in 
the decoding side can be expanded for use. 

ThcA thirteenth aspect of the invention resides in a 
coding and decoding apparatus having any one of the ninth 
through eleventh features and is characterized in that the 

15 identification code of a basic decoding scheme and the 
information that one or some kinds of functional tools will not 
be used, are transmitted as the coding information so that the 
decoding scheme incorporated in the decoding apparatus can be 
simplified for use. 

2 0 ThcA fourteenth aspect of the invention resides in a 

coding and decoding apparatus heaving any one of the ninth 
through eleventh features and is characterized in that the 
identification code of a basic decoding scheme and the 
information that one or some kinds of functional tools will be 

25 replaced with another or others, are transmitted as the coding 
information so that the decoding scheme incorporated in the 
decoding apparatus can be modified for use. 

TheA fifteenth aspect of the invention resides in a coding 
and decoding apparatus having any one of the ninth and tenth 

30 features and is characterized in that when the coding 
information is transmitted, if there are a number of 
combinations of selectable coding information, the combination 
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which minimizes the transmitted amount of information will be 
selected for transmission. 

Next, in a coding and decoding apparatus of the a sixteenth 
aspect of the invention where the coding side transmits coded 
5 data together with identifying information for identifying the 
means of decoding the coded data, and the decoding side is 
capable of storing a plurality of decoding schemes so as to 
perform decoding based on one of the previously stored schemes, 
the coding and decoding apparatus is characterized in that: 

10 before transmitting the coded data to the decoding apparatus, 
the coding apparatus transmits the tools constituting an 
algorithm as the means of decoding the coded data, and the 
decoding apparatus reconstructs the algorithm using the tools 
so as to decode the received coded data based on the algorithm 

15 and stores the tools therein; when the decoding apparatus 
receives the coded data which has been coded by the same tools, 
the decoding side decodes the coded data using the tools 
previously stored and the tools are defined in a hierarchical 
manner so that in place of a tool for a certain rank, the 

2 0 higher-ranked tool can be used to secure the minimum quality of 
the operation; and the coding apparatus on the transmitting 
side simultaneously transmits the decoding tool information and 
the coded data if the decoding apparatus on the receiving side 
does not have the hag — net — decoding tool requested by the 

25 transmitting side. 

Thc A seventeenth aspect of the invention resides in a 
coding and decoding apparatus having the sixteenth feature and 
is characterized in that when the decoding apparatus on the 
receiving side has no decoding tool requested by the coding 

30 apparatus on the transmitting side, the transmitting side 
temporarily changes the coding scheme using the coding tool 
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that is in conformity with the decoding tool present on the 
receiving side. 

i £heAn eighteenth aspect of the invention resides in a 
coding and decoding apparatus having the sixteenth feature and 
5 is characterized in that when the decoding apparatus on the 
receiving side has no decoding tool requested by the coding 
apparatus on the transmitting side, the receiving side, whilst 
downloading the decoded tool transmitted from the transmitting 
side to construct the requested decoding tool, temporarily 
10 decodes the coded data using a substitutable higher-ranked tool 
which is lowered in quality but still is able to perform 
decoding . 

ThcA nineteenth aspect of the invention resides in a 
coding and decoding apparatus having the eighteenth feature and 

15 is characterized in that after the decoding tool requested has 
become prepared, the receiving side starts the decode operation 
using the requested decoding tool . 

According to the invention, the most suitable tools are 
selected based on the comparison between the tool -correspondent 

20 information stored in the tool -correspondent information 
otoring — mcano storage and the received tool -correspondent 
information. Thus, it becomes possible to perform the decoding 
operation efficiently and quickly. Particularly, an effective 
decoding operation is attained because the processing 

25 capacities for the tools are used as the tool -correspondent 
information and the tools, whose capacities fall within a 
permissible range for the decoding operation, are then 
selected. In this case, by transmitting the processing 
capacity which is numerically represented, it becomes possible 

30 to efficiently make a quick comparison in capacities. 

Further, it becomes possible to perform a quick selection 
of the tools because, by making the tool -correspondent 
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information fee— easy to compare, the tools of themselves do not 
need to be compared. Especially, the comparison and selection 
of the tools in the tool storage mcano can be performed quickly 
and effectively by allotting a unique key to each tool as the 
5 tool -correspondent information. 

Since the response control mcano cont roller is provided, 
only when there is a necessity for a tool, it is possible to 
cause the coding apparatus on the other side to transmit the 
information of the tool. Thus, it is possible to shorten the 

10 time for transmission and reception and thereby perform the 
process efficiently . 

Further, since the coding and decoding tools are defined 
by hicrarchigation hierarchically , and the data which conforms 
to the processing capacities of the thus 

15 hicrarchizcd hierarchically defined tools is attached with 
headers for identifying the processing capacity of the tool on 
the coding side, the data can be reconstructed so as to have a 
data structure which conforms to the processing capacity of the 
decoding side and can be decoded . As a result , even if the 

20 processing capacities of the coding and decoding tools are not 
compatible, it is possible to decode the coded data. 

It becomes no more necessary for the decoding side to have 
all of various tools in order to deal with the data which is 
coded by such an algorithm having various coding and decoding 

25 tools as a next -generation coding standard represented by 
MPEG4 . Therefore, it is possible to reduce the cost of the 
apparatus . 

Further, since it is possible to alleviate the restriction 
on the hardware of the decoding apparatus, it becomes possible 
30 for a simple low-cost apparatus to decode the data coded based 
on the next-generation coding standard. 
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In accordance with the scheme described above, the coding 
information, within the coding data stream transmitted from the 
coding apparatus, used for decoding the coded data is made up 
of only the identification code of the basic decoding scheme 
5 and the differential information with respect to the basic 
decoding scheme. Expanded decoding schemes from the basic 
decoding scheme can be represented by the differential 
information. That expansion of the basic decoding scheme is 
formed by designating the functional tools to be added to, 

10 canceled from or replaced in the basic decoding scheme. 
Further, if there are two or more methods in designating the 
differential information, the designating method which requires 
less amount of information is selected so that the amount of 
data to be transmitted becomes minimum. 

15 Further, in the coding and decoding apparatus described 

above, even if the receiving side has not tool having requested 
quality at the time of the first transmission, the transmitting 
side starts coding and the receiving side starts decoding using 
other tools which are lower in quality as compared with the 

20 requested tools but still replaceable. The transmitting side 
simultaneously transmits the requested tools and the thus 
obtained coded data to the receiving side. The receiving side 
downloads the requested tools and prepares the tools whilst 
decoding the coded data using the other tools which are lower 

25 as compared with the requested tools in quality but still 
replaceable. When the establishment of the requested decoding 
tools is completed, the image communication with high quality 
requested will start. 



BRIEF DESCRIPTION OF THE DRAWINGS 

30 The present invention will become more fully understood 



from the detailed description given herein below and the 



accompanying drawings which are given by way of illustration 
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only, and thus, are not limitative of the present invention 
and wherein: 

Fig.lA is a conceptual view showing the structure of a 
coding data stream which is formed by coding (compressing) 
5 image data based on the H.261 scheme; 

Fig. IB is a conceptual view showing the structure of a 
coding data stream which is formed by coding (compressing) 
image data based on a coding scheme such as MPEG4 etc. whose 
algorithm is flexible; 
10 Fig.lC is a block diagram showing a conventional coding 

and decoding apparatus; 

Fig. 2 is a block diagram showing the configuration of 
coder based on H.261; 

Fig. 3 is a block diagram showing the configuration of 
15 decoder based on H.2 61; 

Fig. 4 is a block diagram showing the structure of a coder 
which is provided with several kinds of algorithms; 

Fig. 5 is a block diagram showing the structure of a 
decoder which is realized by using a dedicated hardware device 

2 0 with software; 

Fig. 6 is a block diagram showing the structure of a 
decoder composed of a general -purpose processor and a compiler; 

Fig.7A is a diagram showing an arrangement of pixel data 
obtained by means of an inter frame predictive coding tool of 
25 sampling per single pixel; 

Fig.7B is a diagram showing an arrangement of pixel data 
obtained by means of an inter frame predictive coding tool of 
sampling per M pixel; 

Fig.7C is a diagram showing an arrangement of pixel data 

3 0 obtained by means of an inter frame predictive coding tool of 

sampling per y A pixel; 
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Fig.8A is a diagram showing display images of a pattern TA 
obtained by decoding coded data based on the inter frame 
predictive coding of sampling per single pixel; 

Fig.8B is a diagram showing display images of a pattern TA 
5 obtained by decoding coded data based on the inter frame 
predictive coding of sampling per % pixel; 

Fig.8C is a diagram showing display images of a pattern TA 
obtained by decoding coded data based on the inter frame 
predictive coding of sampling per V* pixel; 
10 Fig. 9 is a structural diagram showing a coding data stream 

which has coding information; 

Fig. 10 is a structural diagram showing a coding data 
stream which uses a decoding scheme identification code for 
coding information; 
15 Fig. 11 is a block diagram showing the first embodiment of 

a coding and decoding apparatus in accordance with the 
invention; 

Fig. 12 is an illustrative diagram showing the operation of 
the coding and decoding apparatus of the first embodiment ; 
20 Fig. 13 is a diagram showing an example of a key 

assignment; 

Fig. 14 is a block diagram showing a coding apparatus in a 

coding and decoding apparatus of the second embodiment in 

accordance with the invention; 
25 Fig. 15 is a block diagram showing a decoding apparatus in 

a coding and decoding apparatus of the second embodiment in 

accordance with the invention; 

Fig. 16 is an illustration showing a coding apparatus and a 

decoding apparatus which are connected through a network; 
30 Fig. 17 is a flowchart showing the steps of communication 

when a coding apparatus and a decoding apparatus are connected; 
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Fig. 18 is a system block diagram showing a configuration 
of an overall transmitting and receiving system which is 
applied to the third embodiment of a coding and decoding 
apparatus of the invention; 
5 Fig. 19 is a conceptual view for explaining a hierarchical 

structure of a coded data obtained using an inter frame 
predictive coding tool of sampling per % pixel; 

Fig. 2 0 is an operational flowchart for explaining the 
operation of inter frame predictive coder; 
10 Fig.21A is a diagram showing the data structure of a 

bottom- ranked coded data which is added with headers in 
conformity with the rank of the inter frame predictive coding 
tool used; 

Fig. 2 IB is a diagram showing the data structure of Fig.21A 
15 in which headers are attached only to the leading part of pixel 
data; 

Fig. 22 is an operational flowchart for explaining the 
operation of an inter frame predictive decoder; 

Fig. 23 is a diagram showing the structure of a coding data 

2 0 stream using a decoding scheme identification code and 

differential information in accordance with the f orth f ourth 
embodiment ; 

Fig. 24 is a structural diagram showing an example of the 
coding apparatus of the fourth embodiment; 
25 Fig. 25 is a structural diagram showing an example of the 

decoding apparatus of the fourth embodiment; 

Fig. 26 is a structural diagram of a system showing the 
fifth embodiment of a coding and decoding apparatus of the 
invention; 

3 0 Fig. 27 is a flowchart for explaining the operation on the 

transmitting side in a coding and decoding apparatus of the 
fifth embodiment of the invention; and 
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Fig. 28 is a flowchart for explaining the operation on the 
receiving side a coding and decoding apparatus of the fifth 
embodiment of the invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
5 The embodiments of the present invention will hereinafter 

be described with reference to the accompanying drawings. 

(First embodiment) 

Fig. 11 is a block diagram showing the first embodiment of 
a coding and decoding apparatus in accordance with the 

10 invention. This coding and decoding apparatus includes a 
controller 11a, a coder 12a, a response controller 13a, a 
decoder 14a, a tool storage controller 15a, a tool storage 
section 16a, a coding capacity storing section 17a, a decoding 
capacity storing section 18a and a capacity comparator 19a. 

15 Controller 11a is a unit for controlling the entire 

apparatus. The total amount of the processing capacities for 
the tools in decoder 14a which constitute the algorithm is 
stored previously in decoding capacity storing section 18a as a 
decoding capacity. The processing capacity for each tool in 

2 0 decoder 14a is represented numerically. Coder 12a transmits 
each tool together with its numerical processing capacity 
required for the tool to the other terminal device. 

Here, the capacity for a tool is represented numerically 
by selecting a standard operation step, determining the 

25 processing capacity for performing the standard operation step 
and comparing the processing load and capacity for the 
operation step of the tool with that of the standard operation 
step . 

The capacity for a tool is not numerically rated by coder 
30 12a whenever the tool is transmitted but is designated 
previously by the implementer or by the selector based on its 
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processing load. The numerical capacity is sent from controller 
11a to the response controller where it is multi-processed with 
coded data and tool information so as to be transmitted. 

Coding capacities received by decoder 14a are stored into 
5 coding capacity storing section 17a by individual tools which 
constitute the algorithm. The output from decoding capacity 
storage section 18a and the sum of the processing capacities 
for the tools constituting the algorithm are both inputted to 
capacity comparator 19a. 

10 Capacity comparator 19a compares the two values so as to 

judge whether the operation of the algorithm using the tool 
received is permissible in the apparatus. The judgment can be 
made by means of a simple big-or-small comparator. If the 
operation can be determined as allowable from the comparison of 

15 the processing capacity, a permissible signal is sent out to 
tool storage controller 15a and the tool received is stored in 
tool storage section 16a so that it can be used for decoding 
the data which follows. 

If, from the comparison of the processing capacity, the 

2 0 decoding capacity of the apparatus is determined to be low to 

store the tool or the operation of the tool is determined as 
impermissible, the received tool is not useful so that a 
storage impermissible signal is sent out to tool storage 
controller 15a so as to prohibit the storage of the tool into 
25 tool storage section 16a. At the same time, instructions for 
transmitting the confirmation of whether the received tool can 
be downloaded for use are sent out to response controller 13a. 

In general, in a coding scheme wherein the classification 
of the tools constituting its algorithm is defined, the 

3 0 decoding capacities on both sides are exchanged at the initial 

stage of the communication so that coding of data and 
transmission will be started after obtaining the decoding 
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capacities of the other terminal devices. However, in a scheme 
wherein coded data is transmitted from the coding side together 
with tools constituting the algorithm for decoding the coded 
data, the load for the decoding process will remarkably vary 
5 depending upon the classification of the selected tools. 
Therefore, it is not considered that the capacity exchange of 
this kind is useful for previously obtaining the exact decoding 
capacity of the other terminal end. Therefore, in the latter 
scheme (the scheme of the invention) , whenever a new tool is 

10 received, the capacity for the tool will be checked so as to 
determine whether the tool can be handled and the determined 
result will be transmitted to the opposite terminal. 

Fig. 12 is an illustrative diagram showing an operational 
example of the invention. Now, the operations of a coding 

15 apparatus 20a and a decoding apparatus 21a a will be explained. 
Here, coding apparatus 2 0a and decoding apparatus 21a both are 
identical or composed of the coding and decoding apparatus 
shown in Fig. 11 and function as coding and decoding 
apparatuses , respectively. 

2 0 First, it is assumed that coding apparatus 2 0a transmits a 

tool f a' and its coding capacity ' Ca 1 required for handling 
tool 'a' to the decoding apparatus. Decoding apparatus 21a 
stores the processing capacity Ca for the tool transmitted into 
coding capacity storage section 17a. The capacities for the 

25 tools constituting the algorithm are stored by individual 
tools. If the sum of the capacities other than Ca is assumed to 
be Cz , the capacity required for the algorithm using the tool 
having capacity Ca can be represented by Ca + Cz. 

This value is compared with a decoding capacity ! Cr' which 

30 is previously determined in decoding apparatus 21a. In this 
comparison, if Ca + Cz < Cr, then tool ! a' which has been just 
received is determined as permissible in the decoding side. If 
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the tool can be used, it is downloaded and a signal indicating 
the completion of the download will be transmitted to the other 
side apparatus- Thus, coding apparatus 20a will be able to 
recognize that data using the tool transmitted therefrom can be 
5 transmitted. 

Next, it is assumed that coding apparatus 20a transmits a 
tool 'b' and its coding capacity ! Cb ! required for handling 
tool f b I to the decoding apparatus. Decoding apparatus 21a 
stores the processing capacity Cb for the tool transmitted into 

10 coding capacity storage section 17a. The capacities for the 
tools constituting the algorithm are stored by individual 
tools. In the same manner as the above operation, the capacity 
required for the algorithm using the tool having capacity Cb 
can be represented by Cb + Cz . 

15 This value is compared with decoding capacity 'Cr' which 

is previously determined. In this comparison, if Cb + Cz > Cr, 
then the tool which has been just received is determined as 
impermissible in the decoding side. In this case, the download 
of the received tool will not be performed and a signal 

2 0 indicating the download error will be transmitted to the other 
side apparatus. Thus, the coding apparatus will be able to 
recognize that data using the transmitted tool cannot be 
transmitted. In response to this, the coding apparatus will 
send another tool in place, or will transmit data using tool 

25 'a' which has been transmitted already so that the data can be 
decoded . 

As described above, the processing capacities for tools 
are used as the tool -correspondent information, and based on 
the processing capacities, tools are selected so that the 
30 capacities for the selected tools fall within a permissible 
range for the decoding operation. Thus, it becomes possible to 
efficiently perform the decoding operation. In this case, by 
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transmitting the processing capacity which is numerically 
represented, it becomes possible to efficiently make a quick 
comparison in capacities. Further, since whether a received 
tool is permissible to use is judged based on the capacity of 
5 the tool and only the tool which is usable will be downloaded, 
it is possible to shorten the processing time required for the 
transmission and reception. 

( Second embodiment ) 

Table 2 shows an example of the control of tools in the 

10 second embodiment of a coding and decoding apparatus in 
accordance with the invention. As shown in Table 2, each tool 
is allotted with a unique key. For example, a tool for motion 
compensation with integer-pixel precision is assigned with a 
key AaOl. In the same manner, all the tools are allotted with 

15 corresponding keys so that each tool can be discriminated. 

Fig. 13 shows an example of the key assignment. For 
instance, the key has a capital alphabetical character for 
broad classification (motion compensated prediction, DCT, 
etc.), a lowercase alphabetical character for subdivision 

20 (integer-pixel precision, half -pixel precision, etc.) and two 
digit numerals for version. 



Table 2 



Key 


Tool 


AaOl 


Integer-pixel precision j 
motion compensation 


AbOl 


Half -pixel precision 
motion compensation 


BaOl 


DCT 


BbOl 


Wavelet transform 


CaOl 


Linear quantization 


CbOl 


Vector quantization 



Fig. 14 is a block diagram showing a coding apparatus of 
2 5 the second embodiment in a coding and decoding apparatus in 
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accordance with the invention. This coding apparatus includes: 
a controller 31 for controlling the entire apparatus; a tool 
storage section 33 for storing tools; a key table 32 for 
storing the memory locations in tool storage section 33 wherein 
5 the processing programs for the tools corresponding to 
different keys are stored; a processing unit 34 which reads out 
a processing program for a tool from the tool storage section 
and executes it; a work memory 3 5 for processing unit 34; and a 
network interface 3 6 for performing communications with the 
10 outside. 

Fig. 15 is a block diagram showing a decoding apparatus of 
the second embodiment in the coding and decoding apparatus in 
accordance with the invention. This decoding apparatus 
includes: a controller 41 for controlling the entire apparatus; 
15 a key buffer 47 for temporarily storing a key received; a tool 
storage section 43 for storing tools; a key table 42 for 
storing the memory locations in tool storage section 43 wherein 
the processing programs for the tools corresponding to 
different keys are stored; a processing unit 44 which reads out 

2 0 a processing program for a tool from the tool storage section 

and executes it; a work memory 45 for processing unit 44; and a 
network interface 4 6 for performing communications with the 
outside . 

Table 3 is an example of stored states of the tools in 
25 tool storage section 33 in the coding apparatus of Fig. 14 as 
well as in tool storage section 43 in the decoding apparatus of 
Fig. 15. Table 4 shows an example of the correspondence of each 
key in key table 32 of Fig. 14 or in key table 42 of Fig. 15 to 
the memory location in tool storage section 33 or 43 for the 

3 0 tool corresponding to the key. 

The coding apparatus of Fig . 14 and the decoding apparatus 
of Fig. 15 can be connected through a network such as an ISDN, a 
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mobile communication network, etc. by means of respective 
network interfaces 36 and 46, as shown in Fig. 16. 

Table 3 



File No. 


Tool j 


0 


Integer- pixel precision 
motion compensation 


1300 


Half -pixel precision 
motion compensation 


2500 


DCT 


4100 


Wavelet transform 


6000 


Linear quantization ! 


7700 


Vector quantization j 



5 

Table 4 



Key 


Storage address 


! AaOl 


0 


AbOl 


1300 


BaOl 


2500 


BbOl 


4100 


CaOl 


6000 


CbOl 


7700 



Fig. 17 is a flowchart showing an example of the order of 
10 the communication steps when the coding apparatus of Fig. 14 and 
the decoding apparatus of Fig. 15 are connected in the manner 
shown in Fig. 16. First, an algorithm used in the coding 
apparatus as well as tools for constructing the algorithm is 
determined (Step SI) . Then, the keys for the tools are 
15 transmitted to the decoding apparatus (Step S2) . The decoding 
apparatus receives the keys and picks them up into key buffer 
47 (Step Tl) . The keys taken in to buffer 47 are compared to 
key table 42 (Step T2) . If some of them are not registered, a 
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request for tool transfer together with the keys is sent out to 
the coding apparatus (Step T3) . The coding apparatus transmits 
the processing programs for the tools corresponding to the keys 
(Step S4) . The decoding apparatus stores the processing 
5 programs for the tools in tool storage section 43 and registers 
the storage addresses with the keys into key table 42 (Step 
T5) . Then, the decoding apparatus transfers the tools from tool 
storage section 43 to processing unit 44 with reference to key 
table 42 (Step T6) . Next, the decoding apparatus sends out a 

10 request for coded data transfer to the coding apparatus (Step 
T7) . Receiving the request, the coding apparatus transmits 
coded data to the decoding apparatus (Steps S5 and S6) . The 
decoding apparatus receives the coded data (Step T8) and 
decodes it (Step T9) . When the coding apparatus has completed 

15 the transmission of all coded data, it sends out a signal 
indicating the end of transmission of coded data to the 
decoding apparatus (Step S7 and S8) . When the decoding 
apparatus receives the signal (Step T10) , the communication is 
ended . 

2 0 In the above way, by using a unique key for each tool as 

an identifier for comparison, it become possible to select the 
required tools in a simple manner, thus making it possible to 
reduce the processing time. Further, keys are used to judge 
whether the same tools are registered in the decoding 

2 5 apparatus, and only when there is no corresponding tool, the 

tool is transferred. Consequently, it is possible to reduce 
the processing time for transmission and reception. 

As has been described, in accordance with this embodiment 
of the invention, the tool -correspondent information stored in 

3 0 the tool -correspondent information storing — mcano storage is 

compared with the received tool -correspondent information so as 
to comparatively select the most suitable tools. Accordingly, 
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it is possible to perform the decoding operation at a high 
efficiency in a short time. Particularly, the processing 
capacities for tools are used as the tool -correspondent 
information, and based on the processing capacities, tools are 
5 selected so that the capacities for the selected tools fall 
within a permissible range for the decoding operation. Thus, 
it becomes possible to efficiently perform the decoding 
operation. In this case, by transmitting the processing 
capacity which is numerically represented, it becomes possible 

10 to efficiently make a quick comparison in capacities. 

Further, since a simple and easily comparable identifier 
is used as the tool -correspondent information, it becomes 
possible to select required tools quickly without examining the 
content of tools. In particular, by allotting a key unique to 

15 each tool as the tool -correspondent information, it is possible 
to efficiently perform the process of comparatively selecting a 
tool from the tool storage means in a short time. 

Since the response control — mcano cont roller is provided, 
only when there — io a ncccnoity for a tool, it is possible to 

2 0 cause the coding apparatus on the other side to transmit the 
information of the tool only when there is a necessity for the 
tool. Thus, it is possible to shorten the time for 
transmission and reception and thereby perform the process 
efficiently. 

2 5 (Third embodiment) 

In reference to drawings, description will be made of a 
coding and decoding apparatus of the third embodiment of the 
invention hereinbelow in which a video signal is sampled per M 
pixel by the inter frame predictive coding so that the coded 

3 0 data is transmitted and the transmitted data is decoded. 
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Fig. 18 is a system block diagram showing the configuration 
of an overall transmitting and receiving system to which a 
coding and decoding apparatus of the invention is applied. A 
transmitting system la shown in the figure includes: a video 
5 input unit 101 converting video information inputting through 
unillustrated visual sensor into a video signal; a coding 
apparatus 102 coding the inputted video signal from the video 
input unit 101 to produce coded data; a transmitter 103 
output ting the coded data produced by coding apparatus 102 onto 

10 a communication net 300. 

Here, coding apparatus 102 is composed of: a transformer 
121 receiving the video signal and subjecting it to the DCT 
(Discrete Cosine Transform) ; a quantizer 122 quantizing the DCT 
coefficients obtained by the DCT in transformer 121; and inter 

15 frame predictive coder 123 receiving the video data quantized 
through quantizer 122 and subjecting it to an inter frame 
predictive coding process. Inter frame Frame predictive coder 
123 has an inter frame predictive coding tool ECi which samples 
the data per single pixel, an inter frame predictive coding 

20 tool EC 2 which samples the data per half pixel, and an inter 
frame predictive coding tool EC 4 which samples the data per 
quarter pixel, so that one of these inter frame predictive 
coding tools will be selected in accordance with the quality of 
display image required on the decoding side. 

25 In the case of transmitting system la shown in Fig. 18, 

inter frame predictive coding tool EC 4 of sampling per M pixel 
is selected in inter frame predictive coder 123. That is, the 
coding tool which provides the highest quality of display image 
is selected . 

30 A receiving system 2a is composed of: a receiver 201 

serving as the input interface for picking up the coded data of 
an image on communication net 300 which has been transmitted 
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from transmitting system la; a decoding apparatus 2 02 receiving 
the coded data from receiver 2 01 and decoding it into a video 
signal; and video output unit 203 receiving the video signal 
from decoding apparatus 2 02 and displaying the video 
5 information on unillustrated monitor. Here, decoding apparatus 
202 is composed of: an inter frame predictive decoder 221 
subjecting the coded data to an inter frame predictive decoding 
process to produce image data; inverse quantizer 222 inverse 
quantizing the image data obtained by decoding from inter frame 

10 predictive decoder 221 and gctt ing providing DCT coefficients; 
and an inverse transformer 223 transforming the DCT 
coefficients produced by inverse quantizer 222 into a video 
signal. In the case of receiving system 2a shown in Fig. 18, 
inter frame predictive decoder 221 has only an inter-frame 

15 predictive decoding tool DC 2 which samples the data per half 
pixel . 

The operations of coding apparatus 102 and decoding 
apparatus 2 02 will be described hereinbelow by explaining the 
operations of transmitting system la and receiving system 2a 

2 0 thus configurated. 

First, the operation of transmitting system la will be 
described. Video input unit 101 converts video information 
inputted through a visual sensor such as a camera etc. into a 
video signal and supplies it to coding apparatus 102 . The 

2 5 video signal inputted to coding apparatus 102 is converted into 
image data through transformer 121 and quantizer 122. Inter 
frame Frame predictive coder 123 receives the image data and 
subjects it to an inter frame predictive coding process to 
produce coded data. This coded data is supplied to transmitter 

30 103. Transmitter 103 outputs the coded data onto communication 
net 3 00. 
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Here, inter frame predictive coder 123 produces coded data 
in a hierarchical structure in accordance with the coding 
capacity of the inter frame predictive coding tool used for 
coding. Before the detailed description of the operation of 
5 this inter frame predictive coder 123, the hierarchization of 
coded data will be explained. In hierarchizing coded data, the 
ranks of the inter frame predictive coding tools used for 
producing coded data are to be defined. That is, when the 
coded which is obtained by the first inter frame predictive 

10 coding tool for a certain rank contains the coded data which is 
obtained by the second inter frame predictive coding tool for 
another rank, the rank of the former inter frame predictive 
coding tool is defined to be lower than the rank of the latter 
inter frame predictive coding tool. According to this 

15 definition, the inter frame predictive tools of sampling per 
single pixel, per % pixel and per X A pixel belong to the top, 
medium and bottom ranks, respectively, based on the 
inclusionary relation of the coded data. 

Fig. 19 is a conceptual view for explaining the 

20 hierarchical structure of the coded data obtained using the 
inter frame predictive coding tool of sampling per M pixel. In 
the figure, top-rank coded data D r i/ medium- ranked coded data 
D r2 , and bottom- ranked coded data D r 4 are produced by the 
aforementioned top-ranked, medium- ranked and bottom-ranked 

25 inter frame predictive coding tools, respectively. Pixel data 
ai to di, a 2 to i 2 , and a 4 to y 4 shown in Fig.7C correspond to 
top-ranked coded data D r i, medium- ranked coded data D r2 and 
bottom- ranked coded data D r4 , respectively. As understood from 
Fig. 19 and Figs. 7A to 7C, the bottom-rank coded data has such 

3 0 a data structure that it hierarchically includes the medium and 
top-ranked coded data. The medium- ranked coded data has such a 
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data structure that it hierarchically includes the top-ranked 
coded data. 

The hierarchization of coded data by the inter- frame 
predictive coding process is obtained by noticing the 
5 hierarchical structure of the coded data as stated above. That 
is, coding apparatus 102 selectively adds a header 
(identifiers) for identifying the rank of data, to each piece 
of pixel data a 4 to y 4 constituting coded data D r4 of the bottom 
rank, so that the coded data whose rank is defined in 

10 correspondence with the coding capacity of the inter- frame 
predictive coding tool used is created. 

Now, the operation of inter- frame predictive coder 123 
will be described in detail with reference to the flowchart 
shown in Fig. 20. First, inter frame predictive coder 123 

15 inputs image data from quantizer 122 (Step SOI) , and then it 
selects an image frame predictive coding tool in accordance 
with the quality of display image on the decoding side (Step 
S02) . Since, in inter frame predictive coder 123, inter frame 
predictive tool EC 4 of sampling per M pixel (the inter frame 

2 0 predictive tool for the bottom rank) which provides high- 
quality image display has been selected as already mentioned, 
the image data is subjected to the inter frame predictive 
coding process by means of inter frame predictive coding tool 
EC 4 (Step S03) . Each piece of image data a 4 to y 4 thus obtained 

2 5 is added with one of headers HDi to HD 4 which corresponds to 
the rank of the inter frame predictive coding tool (Step S04) . 
Thus, coded data D r4 of the bottom rank is produced. Then, as 
the coded data D r4 is outputted to transmitter 103, the 
operation is ended. 

30 In the above Step S02, when the high-quality display image 

is not requested, the inter frame predictive coding process is 
performed by using the inter- frame predictive coding tool of 
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sampling per 34 pixel or single pixel (Step S05 or S07) . 
Thereafter, the thus obtained coded data is added with 
corresponding headers (Step S06 or S08) . 

Now, headers HDi to HD 4 will be explained. Figs. 21A and 
5 2 IB are diagrams showing the data structures of bottom- ranked 
coded data D r4 where each piece of coded data is added with any 
of headers HDi to HD 4 which corresponds to the rank of the 
inter frame predictive coding tool. More specifically, there 
figures show the data structures wherein, from pixel data a 4 to 

10 y 4 shown in Fig.7C, each piece of pixel data a 4 to j 4 arranged 
on the first and second lines is added with one of headers HDi 
to HD 4 . In Fig.21A, header HDi is attached to the pixel data 
(the pixel data of the pixels indicated by ' + ' in Fig.7C) 
corresponding to the data which is produced by the inter- frame 

15 predictive coding of sampling per single pixel. Next, header 
HD 2 is attached to the pixel data (the pixel data of the pixels 
indicated by 1 o' in Fig.7C) corresponding to all the data which 
is produced by the inter frame predictive coding of sampling 
per 34 pixel, except the pixel data produced by the inter frame 

20 predictive coding of sampling per single pixel. Further, head 
HD 4 is attached to the pixel data (the pixel data of the pixels 
indicated by X A' in Fig.7C) corresponding to all the data which 
is produced by the inter frame predictive coding of sampling 
per M pixel, except the pixel data produced by the inter frame 

25 predictive coding of sampling per 34 pixel. 

In this way, which each piece of pixel data constituting 
coded data D r4 is attached with the header corresponding to the 
rank of the inter-frame predictive coding tool, the set of 
pixel data added with header HDi becomes equal to the content 

3 0 of the coded data which is obtained by inter frame predictive 
coding tool ECi of the top rank. The set of pixel data added 
with header HDi or HD 2 becomes equal to the content of the 
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coded data which is obtained by inter- frame predictive coding 
tool EC 2 of the medium rank. Therefore, on the decoding side, 
pixel data a x to di, a 2 to i 2 and a 4 to y 4 which correspond to 
respective inter- frame predictive coding tools of different 
5 ranks, among coded data D r4 can be discriminated by identifying 
the class of headers attached to pixel data a 4 to y 4 
constituting coded data D r4 . Further, it is possible to 
identify the coding tools used for producing coded D r4 by 
examining the classes of the headers. In this way, coded data 

10 D r4 of the bottom rank which is constituted by the pixel data 
stream with headers attached is sent out onto communication net 
300 by means of transmitter 103. 

As shown in Fig. 2 IB, when a series of pixel data is added 
with the same headers, the headers may be attached to only the 

15 leading part of the pixel data. This attachment of headers 
will inhibit the increase of the amount of data due to the 
addition of headers . 

Next, the operation of receiving system 2a will be 
described. Receiver 201 as the input interface of receiving 

20 system 2a picks up coded data D r4 from communication net 300 
and supplies it to decoding apparatus 202. Coded data D r4 
taken in to decoding apparatus 202 is decoded in inter- frame 
predictive decoder 221 to be converted into image data. The 
thus obtained image data after the conversion is further 

2 5 converted into a video signal through inverse quantizer 222 and 
inverse transformer 223. Video output unit 203 develops the 
video signal into a visual image as video information on the 
monitor. Here, inter frame predictive decoder 221 has only 
inter frame predictive decoding tool DC 2 of sampling per % 

30 pixel which has the decoding capacity corresponding to coding 
capacity of inter-frame predictive coding tool EC 2 . This 
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inter frame predictive decoding tool DC 2 is defined as a tool 
of the medium rank. 

The operation of inter frame predictive decoder 221 will 
hereinbelow be described in detail with reference to the 
5 operational flowchart shown in Fig. 22. Inter f r amc Frame 
predictive decoder 221 receives coded data D r4 from receiver 
2 01 (Step Sll) . Then, it is judged whether the rank (coding 
capacity) of the coding tool of the inter frame predictive 
coding tool used in coding is equal to the rank (decoding 

10 capacity) of inter frame predictive decoding tool DC 2 equipped 
in inter frame predictive decoder 221 (Step S12) . This 
judgment can be made by identifying header HD 4 among headers 
HD1 to HD4 attached to the coded data. Since input coded data 
D r4 is of the bottom rank, the rank of the input coded data is 

15 determined not to be identical with the rank (medium rank) of 
decoding tool DC 2 equipped in inter frame predictive decoder 
221 (Step S12, No) . 

Next, it is judged whether the rank of coding tool EC 4 
(bottom-ranked) which has been used to produce coded data D r4 

20 is lower than the rank of decoding tool DC 2 (medium- ranked) 
equipped in inter frame predictive decoder 221 (Step S15) . In 
this judgment, if it is on the lower side (Step S15, Yes), only 
the pixel data which has been coded with headers HDi and HD 2 
which conform with the decoding capacity of inter frame 

25 predictive decoding tool DC 2 of the medium rank, is extracted 
from coded data D r4 to reconstruct coded data (Step S16) . 

Specifically, from pixel data a 4 to y 4 obtained by the 
inter frame predictive coding process of sampling per M pixel, 
the pixel data which corresponds to the coded data obtained by 

3 0 the inter frame predictive coding of sampling per Yt pixel is 
picked up to reconstruct coded data (Step S16) . The extraction 
of data is performed by identifying the header added to each 
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piece of pixel data so that the pixel data added with header 
HDi or HD 2 is extracted from pixel data a 4 to y 4 . Then, the 
headers are removed from thus extracted data, whereby coded 
data Dr4 is reconstructed into coded data D r2 . Coded data D r2 
5 thus reconstructed is made to undergo the inter frame 
predictive decoding by using tool DC 2 of the medium rank (Step 
S17) . 

Suppose that inter frame predictive decoder 221 has 
unillustrated inter frame predictive decoding tool DC 4 of 

10 sampling per X A pixel. In this case, it is determined in the 
judgment at Step S12 that the rank of decoding tool equipped on 
the decoding side is the same as that of the coding tool (Step 
S12, Yes), and headers HDi to HD 4 are removed from coded data 
D r4 , whereby coded data D r4 without headers is reconstructed 

15 (Step S13) . Then, inter frame predictive decoding tool DC 4 is 
used to cause reconstructed coded data D r4 to undergo to the 
inter-frame predictive decoding process. The image data 
obtained from this decoding process is outputted to inverse 
quantizer 222 (Step S14) , and thus the decoding operation is 

2 0 ended. 

Further, if the inter-frame predictive coding is executed 
using inter frame predictive coding tool ECi of sampling per 
single pixel, the received coded data must be coded data D ri 
which belongs to the higher rank. Therefore, in the above 

2 5 judgment at Step S15, the rank of the coding tool is determined 
not to be on the lower side (Step S15, No) . In this case, 
since coded data D r2 which conforms with inter frame predictive 
decoding tool DC 2 equipped in the decoding side can not be 
reconstructed from coded data D ri which is received, the 

30 operation is ended without executing the decoding process. 

As has been apparent from the foregoing description, in 
accordance with the coding and decoding apparatus of the third 
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embodiment, n-ranked coded data which is produced using an n- 
ranked coding tool and decoded using an n-ranked decoding tool 
has a hierarchical structure which includes (n+1) -ranked coded 
data which is produced using a (n+1) -ranked coding tool and 
5 decoded using a (n+1) -ranked decoding tool. This inclusionary 
relation holds for an arbitrary positive integer n. 

On the coding side which has the n-ranked coding tool, the 
coding mcano coder produces the n-ranked coded data using the n- 
ranked coding tool. The identifier adding mcano adder attaches 

10 N-ranked identifiers to N-ranked coded data which is included 
in the n-ranked coded data but is other than (N+1) -ranked coded 
data included in the N-ranked coded data. In this case, N is a 
positive integer which is equal to n or more and whose upper 
boundary is limited depending upon the depth of the hierarchy 

15 of the coded data. As a result, if n = 1, for example, then 
first-ranked identifiers are attached to first-ranked coded 
data except for second- ranked coded data included in the first - 
ranked coded data. The second-ranked identifiers are attached 
to the second-ranked coded data included in the first -ranked 

2 0 coded data except for the third-ranked coded data included in 
the second-ranked coded data. Similarly, the fourth or fifth- 
ranked identifiers are attached to the data in accordance with 
the depth of the hierarchy of the coded data included in the 
first-ranked coded data. 

25 Next, on the decoding side which has a m- ranked decoding 

tool (m is a positive integer satisfying m > n) , the data 
rcconn true ting — mcano reconstructer extracts a set of the N- 
ranked coded data which is included in the n-ranked coded data 
and attached with the N-ranked identifiers where N > m. By 

30 this extraction, if n = 1, m = 2, for example, second-ranked 
coded data included in first -ranked coded data (a set of the 
coded data with second, third-ranked, .... identifiers 
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attached) will be extracted. If n = 1, m = 3 , as another 
example, third- ranked coded data included in the first coded 
data (a set of the coded data with third, fourth- ranked, . . . 
. identifiers attached) will be extracted. From the thus 
5 extracted set of the N-ranked coded data with N-ranked 
identifiers attached, the m-ranked coded data is reconstructed. 
Then, the decoding mcano decoder decodes the m-ranked coded data 
using the m-ranked decoding tool. Here, it is possible to 
perform the coding and decoding operations using an inter frame 

10 predictive coding tool for the coding tool and an inter frame 
predictive decoding tool for the decoding tool. 

As stated above, since the coding and decoding tools are 
defined by hierarchization, and the data which conforms to the 
processing capacities of the thus hierarchized tools is 

15 attached with headers for identifying the processing capacity 
of the tool on the coding side, the data can be reconstructed 
so as to have a data structure which conforms to the processing 
capacity of the decoding side and can be decoded. As a result, 
even if the processing capacities of the coding and decoding 

20 tools are not compatible, it is possible to decode the coded 
data . 

It becomes no more necessary for the decoding side to have 
all of various tools in order to deal with the data which is 
coded by such an algorithm having various coding and decoding 
25 tools as a next -generation coding standard represented by MPEG 
4. Therefore, it is possible to reduce the cost of the 
apparatus . 

Further, since it is possible to alleviate the restriction 
on the hardware of the decoding apparatus, it becomes possible 
3 0 for a simple low-cost apparatus to decode the data coded based 
on the next -generation coding standard. 

( Fourth embodiment ) 
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The fourth embodiment of the invention will be described 
hereinbelow with reference to drawings. 

Fig. 23 is a diagram showing an example of the structure of 
a coding data stream used in the coding and decoding apparatus 
5 of the invention. 

In Fig. 23, a decoding scheme identification code 431 as a 
component of the coding information is the same as that of 
Fig. 10, and is an identification code for a previously defined 
basic decoding scheme. Additional condition information 432 

10 and tool information 433 are instructional information for 
adding, deleting or replacing certain procedures for the basic 
decoding scheme identification code 431. That is, when, for 
example, the coding scheme which, although it basically 
conforms to H.261, is partially modified at minute points is 

15 used, only the decoding scheme identification code of H.261 
with the differential information from the scheme is 
transmitted as the coding information in order to enable the 
reconstruction of the decoding scheme on the decoding side. In 
this way, it is possible to eliminate the loss of transmitting 

20 all the functional tool information as shown in Fig. 9. 

As stated above, the additional condition information may 
include specifying conditions such as addition, deletion and 
replacement, etc. For example, the transaction of motion 
compensation used in the H.261 coding scheme to efficiently 

25 display the movement of objects, tends to lower the speed of 
processing since a lot of operations are needed on the coding 
side. In order to avoid such a situation, it is possible not 
to use the motion compensation. In such a case, the additional 
condition information can be provided with an instruction of 

3 0 deleting a functional tool while the functional tool 
information can be provided with the identification code of the 
motion compensating tool. In this way, it is possible to 
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drastically reduce the transmitted amount of data compared to 
the case where the revised coding information based on H.261 
which does not have the motion compensating is transmitted from 
the beginning. 

5 Similarly, other instructions such as addition of a 

functional tool which is not included in the basic decoding 
scheme or replacement of a functional tool with another 
functional tool which has a similar function, can be made by 
the combination of the additional condition information and the 

10 functional tool information. Thus, it is possible to reduce 
the transmitted amount of data compared to the case where all 
the information which constitutes the decoding scheme is 
transmitted from the beginning. 

Now, the configurations of the coding apparatus and the 

15 decoding apparatus for practicing the above operations will be 
described hereinbelow. 

Fig. 24 is a block diagram showing an example of the 
embodiment of the coding apparatus having the function stated 
above. In Fig. 24, a reference numeral 441 designates a coding 

2 0 scheme database, which has basic coding schemes and a group of 
functional tools stored therein so that they can be read out 
from the outside. Designated at 442 is a coding scheme 
selector which selects the most preferable coding scheme in 
accordance with the nature of the input data, and reads out the 

2 5 components of a required coding scheme and the content of 
functional tools. The thus obtained coding scheme is 
transferred to a coding section 443 where the input data is 
sequentially coded. The control of the processing tools for 
the aforementioned sections as well as data input/output 

30 control etc., is performed altogether in a coding controller 
444. If, in coding scheme selector 442, there are a number of 
selectable oclccting ways of the coding information representing 
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the determined scheme, the selector compares ia— the transmitted 
amount of data between different Qclccting selectable ways and 
selects the one whose transmitted amount of data is minimized 
and transmits it, in order to avoid the reduction in the 
5 efficiency in the use of the communication line as much as 
possible . 

Fig. 2 5 is a block diagram showing an example of the 
embodiment of the decoding apparatus having the function 
described above. In Fig. 25, a reference numeral 451 designates 

10 a decoding scheme database, which has basic coding schemes and 
a group of functional tools stored therein so that they can be 
read out from the outside. Designated at 452 is a decoding 
scheme interpreter which reads out the components of a required 
decoding scheme and the content of functional tools from 

15 decoding scheme database 451. The thus decoding scheme is 
transferred to a decoding section 4 53 where the input coded 
data is sequentially decoded. The control of the processing 
tools for the aforementioned sections as well as data 
input/output control etc., is performed altogether in a 

20 decoding controller 454. 

As has been apparent from the foregoing description, the 
fourth embodiment of the invention offers the following 
advantages . 

First, since when the coding information is transmitted, 
25 an identification code of a previously defined basic decoding 
scheme and the differential information from the basic decoding 
scheme are transmitted, it is possible to reduce the 
transmitted amount of data compared to the case where the 
information of the functional tools for the decoding scheme and 
3 0 other information are all determined, and the efficiency in the 
use of the communication line will be prevented from being 
deteriorated very much. 
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Next, since when the coding information is transmitted, 
the information that one or some kinds of functional tools will 
be added to the basic decoding scheme-? — 3r& is transmitted 
therewith as the differential information, the decoding scheme 
5 incorporated in the decoding apparatus can be expanded for use. 

Since when the coding information is transmitted, the 
information that one or some kinds of functional tools will not 

be used-7 3r & is transmitted therewith as the differential 

information, the decoding scheme incorporated in the decoding 
10 apparatus can be simplified for use. 

Further, since when the coding information is transmitted, 
the information that one or some kinds of functional tools will 
be replaced with another or others-? — ±-s is transmitted therewith 
as the differential information, the decoding scheme 
15 incorporated in the decoding apparatus can be modified for use. 

Finally, when the coding information is transmitted, if 
there are a number of combinations of selectable coding 
information, the amount of combined information is compared 
with the others so that the combination which minimizes the 
2 0 transmitted amount of information can be selected for 
transmission. Accordingly, it is possible to inhibit the 
efficiency in the use of the communication line from being 
degraded very much. 

(Fifth embodiment) 

25 The fifth embodiment of the invention will be detailed 

with reference with drawings. For the description hereinbelow, 
the inter frame predictive coding and decoding method will 
explained as the illustrative configuration of a hierarchical 
coding and decoding tool. In the description, the method of 

30 sampling per single pixel, the method of sampling per ^ pixel 
and the method of sampling per X A pixel are assumed to be in the 
descending order from the top-rank. 
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Fig. 26 is a system block diagram showing an example of the 
fifth embodiment of the invention wherein lb and 2b designate 
coding and decoding apparatuses and 603 a communication net. 
Coding and decoding apparatus lb includes a video input unit 
5 611, a switch 612, a transformer 613, a quantizer 614, an 
inter-frame predicting coder 615, a transmitter 616, a tool 
information transmitter/receiver 617 and a tool 
information/downloading controller 618. Coding and decoding 
apparatus 2b includes a video output unit 621, a switch 622, an 

10 inverse transformer 623, an inverse quantizer 624, an inter 
frame predicting decoder 625, a receiver 626, a tool 
information transmitter/receiver 627 and a tool 
information/downloading controller 62 8 . 

Now, description will be made of a case where an inputted 

15 video mage is transmitted through communication net 603 from 
coding and decoding apparatus lb to coding and decoding 
apparatus 2b and the received signal is outputted. Fig. 2 7 is a 
flowchart of the operation in coding and decoding apparatus lb. 
Fig. 28 is a flowchart of the operation in coding and decoding 

2 0 apparatus 2b. 

When a video image is inputted to video input unit 611 in 
coding and decoding apparatus lb (Step S21) , the video signal 
is coded through transformer 613, quantizer 614 and inter frame 
predictive coder 615, each having a number of tools. Each of 
25 the tools are previously hierarchized in such a manner that the 
tools which are replaceable with the other tools are defined as 
bottom-ranked tools and the tools which cannot be replaced with 
the other tools are defined as top-ranked tools. In this 
embodiment, as shown in Fig. 26, the tools from the top-rank to 

3 0 the bottom-rank, are designated with A, B and C, for 

convenience. In this example, as shown in Fig. 26, the inter 
frame predictive coding tools are arranged so that the inter 
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frame predictive coding tool of sampling per single pixel (tool 
A) , the inter frame predictive coding tool of sampling per ^ 
pixel (tool B) and the inter frame predictive coding tool of 
sampling per X A pixel (tool C) are from the top to the bottom. 
5 Therefore, each of the tools in transformer 613, quantizer 614 
and inter frame predicting coder 615 are selected through 
switch 612 in conformity with the quality of the coded video 
data to be transmitted from coding and decoding apparatus lb. 
In this embodiment, the inter -frame predictive coding of 

10 sampling per X A pixel is categorized as the highest quality 
while the inter frame predictive coding of sampling per single 
pixel is categorized as the lowest quality. 

In accordance with this example, first, tool C from each 
section is selected as the highest quality tool by switch 612 

15 (Steps S22, S23 and S24) . The inputted video image is processed 
through selected tools C, in particular, the signal is made to 
undergo the inter- frame coding tool of the tool of sampling per 
y A pixel. The thus processed signal is transmitted onto 
communication net 603 (Step S25) . This coding information on 

2 0 communication net 603 is accepted by means of coding and 
decoding apparatus 2b. At that moment, coding and decoding 
apparatus lb acquires the knowledge of the state of coding and 
decoding apparatus 2b from tool information 

transmitter/receptor 617 on the transmitting side and tool 

2 5 information transmitter/receiver 627 on the receiving side, and 
constructs the required tool information in tool 
information/downloading controller 618 (Step S26) . The thus 
obtained tool information is sent out from tool information 
transmitter/receptor 617 whilst the coding information is 

30 transmitted from the transmitter 616 (Step S27) . 

At this point, until the tools in coding and decoding 
apparatus 2b become ready (Step S28) , coding and decoding 
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apparatus lb may temporarily change its tools (Step S29) to 
transmit coded data (Step S30) . Thus, since the transmitting 
side, in place of performing a heavy process using the tools 
for high quality requested, may perform a light-loaded process 
5 using the tools for low quality, and since the receiving side 
need not take the trouble to modify the coded data of high 
quality into coded data of low quality, these facts will 
relieve the burdens on both the apparatuses. In this example, 
this temporal coding is performed using the inter frame 

10 predictive coding tool of sampling per single pixel (Step S24) . 

In this embodiment, in the coding and decoding apparatus 
2b on the decoder, inverse converter 623 and inverse quantizer 
624 have their respective tools C as the decoding tools 
requested, but inter frame predictive decoder 625 dosc does not 

15 have the requested tool of sampling per V* pixel. For this 
reason, as shown in the flowchart in Fig. 27, coding and 
decoding apparatus lb on the transmitting side transmits the 
tool information whilst sending out either the coding 
information of low quality which is prepared by the modified 

20 tools or the coding information of the original quality. As 
show in Fig. 28, coding and decoding apparatus 2b on the decoder 
receives coded data (Step S31) . Since no requested tool is 
found in the decoder (Step S3 2) , the decoding is performed by 
using the top-ranked tools (Step S33) although the quality 

25 becomes degraded from the requested level. At the same time, 
the apparatus 2b receives the tool information of the requested 
tool by tool information transmitter/receiver 627 (Step S34) so 
as to construct the tool in the tool information/downloading 
controller 628 (Step S35 to S37) . 

3 0 In this way, as soon as the requested tool becomes ready, 

the operation is switched from the top-ranked tool to the 
requested tool by means of switch 622 so that the coded data of 
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the requested quality can be decoded thus displaying the video 
output by means of video output unit 621. In this example, 
whilst the coded data which has been coded by the inter frame 
predictive coding of sampling per X A pixel is decoded 
5 temporarily by means of inter frame predictive coding of 
sampling per single pixel, the tool for the inter frame 
predictive coding of sampling per % pixel is downloaded, and as 
soon as the tool becomes prepared, the coded data will be 
decoded by the inter frame predictive coding of sampling per y A 
10 pixel. 

As has been described above, in accordance with the fifth 
embodiment of the invention, when the coded data is 
transmitted, if the tools which barely permit the decoding of 
the data although the decoded data degrades, is found in the 

15 receiving side, the transmitting side transmits the requested 
tools constructing the algorithm as the means of decoding the 
coded data together with the coded data. The transmitting side 
temporal ity temporarily transmits the coded data which is made 
to conform to the tools barely permitting the decoding though 

20 the decoded data degrades, whilst sending out the tool 
information. Further, the receiving side, whilst decoding using 
the tools which barely permit the decoding although the data 
degrades, downloads the requested tools. Moreover, after the 
requested tools become prepared, the receiving side decodes the 

2 5 coded data of the requested quality. 

In conclusion, as in the next-generation image coding 
scheme (MPEG4 etc.), when a variety of coding and decoding 
tools are used in the algorithm, the coding and decoding tools 
which can be replaced by other tools although the quality 

30 degrades if the substitution is used, are defined as the 
bottom- ranked tools for the highest quality and the tools which 
are not replaceable are defined as the top-ranked tools. In 
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this way, the apparatus is hierarchized so that the 
interchangeability is established based on the top-ranked tools 
and the tool information can be downloaded. In such a coding 
and decoding apparatus of the fifth embodiment of the 
5 invention, if the decoding apparatus on the receiving side has 
no tool requested by the transmitting side, the coding 
apparatus on the transmitting side will transmit decoding tool 
information and coded data simultaneously. Therefore, compared 
to the configuration in which only the tool information is 

10 transmitted at first, and the coded data is decoded based on 
the tool only after the tool information is downloaded, it 
becomes possible to save the time during which the tool 
information alone is transmitted. As a result, it is possible 
to quicken the start of transmission of coded data. 

15 Since when the decoding apparatus on the receiving side 

has no decoding tool requested by the transmitting side, the 
coding apparatus on the transmitting side temporarily changes 
the coding scheme using the coding tool that is in conformity 
with the decoding tool present on the receiving side, it is 

2 0 possible to reduce the delay of the start of decoding although 
the quality temporarily degrades. 

When the decoding apparatus on the receiving side has no 
decoding tool requested by the coding apparatus on the 
transmitting side, the receiving side, whilst downloading the 

25 decoding tool transmitted from the transmitting side to 
construct the requested decoding tool, temporarily decodes the 
coded data using a subst itutable top-ranked tool which is 
lowered in quality but still is able to perform decoding. 
Therefore, the top-ranked tool can be used at the early stage 

30 thus making it possible to reduce the delay of the start of 
decoding although the quality temporarily degrades. 
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Moreover, since after the decoding tool requested has 
become prepared, the receiving side starts to decode using the 
requested decoding tool, the quality an be assured after the 
completion of the construction of the requested decoding tool. 
5 The invention being thus described, it will be obvious 

that the same may be varied in many ways. Such variations are 
not to be regarded as a departure from the spirit and scope of 
the invention, and a 1*1 such modifications as would be obvious 
to one skilled in the art are intended to be included within 
10 the scope of the following claims. 



